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Abstract. Deterministic graph grammars generate regular graphs, that form a structural extension of 
configuration graphs of pushdown systems. In this paper, we study a probabilistic extension of regu- 
lar graphs obtained by labelling the terminal arcs of the graph grammars by probabilities. Stochastic 
properties of these graphs are expressed using PCTL, a probabilistic extension of computation tree 
logic. We present here an algorithm to perform approximate verification of PCTL formulae. More- 
over, we prove that the exact model-checking problem for PCTL on probabilistic regular graphs is 
undecidable, unless restricting to qualitative properties. Our results generalise those of (8), on prob- 
abilistic pushdown automata, using similar methods combined with graph grammars techniques. 

1 Introduction 

Formal methods have proven their importance in the validation of hardware and software systems. In or- 
der to represent real systems more accurately, several aspects need to be reflected in the model. Recursion 
and random events are examples of such extra features and lead to complex models that incorporate two 
sources of complexity: probabilities and infinite state space. For each of these features independently, 
verification techniques have been established. 

Infinite state systems, on the one hand, cover a large range of expressive power. Among them push- 
down systems offer a simple infinite framework by extending finite state systems with a stack. Despite 
the fact that their configurations graph is infinite, pushdown systems enjoy several interesting properties. 
In particular, the reachability problem is decidable, and the reachability set is effectively regular 
Moreover, monadic second order logic (MSO) ifTTTl is decidable over the graph of configurations for 
pushdown automata. Alternatively, the configurations graphs of pushdown automata can be generated by 
deterministic graph grammars, introduced by Courcelle Q. Deterministic graph grammars generate reg- 
ular graphs which also have decidable MSO Q, and which characterise the same structures as pushdown 
systems @ when restricting to finite degree. We advocate that these grammars offer a simple presenta- 
tion and emphasize the structural properties of graphs. Indeed, contrary to pushdown automata, graph 
grammars are more robust to transformations. Precisely, many transformations of pushdown automata 
affect the configurations graph, and thus its stucture-based properties. On the contrary, graph gram- 
mars allow for transformations in the representations which preserve the structure. Indeed, most graph 
grammar transformations presented in [ 5 ] preserve, up to isomorphism, the generated graph. Using such 
representations thus seems promising in order to express structural properties of systems. 

Probabilistic systems, on the other hand, also raised intensive research concerning verification, start- 
ing with model-checking algorithms for Markov chains, and Markov decision processes for various log- 
ics. In the last decade, models combining probabilities and infinite-state spaces have been investigated. 
Examples of such models are probabilitic pushdown systems and probabilistic lossy channel systems. 
These systems are finitely described and generate infinite Markov chains on which one can express prob- 
abilistic properties, for example using the probabilistic extension of CTL, PCTL |9). This logic allows 
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to express, e.g., the probability of satisfying a given CTL path formula. More generally, PCTL can be 
seen as a variant of CTL where the usual forall quantifier is replaced with a probabilistic comparison to 
a threshold: the whole state formula is satisfied if the probability of the set of executions satisfying the 
CTL path formula meets the constraint expressed by the threshold. A restricted fragment of this logic, 
called qualitative PCTL is obtained when allowing values and 1 only for the thresholds. In constrast, 
the general case (where threshold values are arbitrary) is referred to as quantitative PCTL. The model- 
checking problem for probabilistic logics over infinite Markov chains generated by probabilistic lossy 
channel systems or probabilistic pushdown automata is a natural and deeply investigated issue. Con- 
cerning probabilistic pushdown automata, a series of papers established fundamental model checking 
results (2][8l [TUMI], some of the most significant ones being the decidability of the model checking of 
qualitative PCTL formulae, and the undecidability of the quantitative version. 

In this paper, we consider a probabilistic extension of regular graphs. To this aim, we define prob- 
abilistic graph grammars as graph grammars where terminal arcs are labelled with probabilities. Prob- 
abilistic graph grammars hence generate infinite-state Markov chains, and form a natural generalisation 
of probabilistic pushdown automata. For these models, we extend the results of [8] concerning the 
model-checking of PCTL. Precisely, for probabilistic graph grammars we prove the decidability of the 
qualitative PCTL model-checking ; we detail how to approximate the probability of path formula ; and 
we prove the undecidability of the exact quantitative PCTL model-checking. 

2 Regular graphs and probabilistic regular graphs 

2.1 Hypergraphs and graphs 

Let F be a ranked alphabet, and p : F — > N its ranking function that assigns to each element of F its arity. 
We denote by F n the set of symbols of arity n. Given V an arbitrary set of vertices, a hypergraph G is a 
subset of \J n >iF n V". The vertex set of G, denoted V G , is defined as the set V G = {v G V \ FV*vV* n G ^ 0}. 
In our setting, this set is always countable. An element of F n V n is an hyperarc of arity n, denoted by 
/ vi v 2 • • • v n . 

Graphs form a restricted class of hypergraphs where hyperarcs have arity at most 2. Precisely, a 
graph G over V is a subset of F2VV U F\V . For a G F2, and s,t G V, ast G G is an arc of G with source s, 
target t and label a. For a £ F\ and s G V, if as is an element of G, a is referred to as the colour of vertex 
s (observe that a vertex may have several colours). Dom(G), Im(G) and Wq denote respectively the set 
of sources, targets and vertices of G. The in-degree (resp. out-degree) of a vertex v is the number of arc 
having source (resp. target) v; its degree is the sum of the in and out-degrees. The transition relation 
underlying G is composed of transitions s — >q t for ast G G. A path in G is a finite sequence of transitions 

at a„-i . , ai"-a„ 

vi — > V2-- ■ > v n , also noted vi => q v„. 

A graph morphism from G to G' , is a mapping g : Vq — > V& such that for all u, v G Vg, u v implies 
g(u) g(v). Such a morphism is an isomorphism if g is a bijection, and its inverse is also a morphism. 

2.2 Graph grammars 

Graph grammars are a convenient tool to represent graph transformations. Starting from a hyperarc, the 
axiom, and using rewriting rules, these grammars generate families of infinite graphs that enjoy interest- 
ing properties (for example the decidability of MSO theory, or the fact that they generate context-free 
languages). Graphs generated by graph grammars form a slight extension of the graphs of configura- 
tions for pushdown automata, namely such a graph may have vertices of infinite degree (still there are 
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only finitely many distinct degrees). A motivation for generating these graphs using graph grammars 
rather than pushdown automata is to emphasize the structural properties of the obtained graphs, since 
they are defined up to isomorphism. In particular, stochastic properties of Markov chains (like probabil- 
ity of a path or a set of paths) are invariant under graph isomorphism, this justifies the use of structural 
characterizations such as graph grammars. 

Definition 2.1. A hypergrctph grammar (HR-grammar for short), is a tuple & = (N, T,R,Z), where: 

• N and T are two ranked alphabets of non-terminal and terminal symbols, respectively; 

• Z € ./V is a 0-arity non-terminal, the axiom; 

• Ris a set of rewriting rules assigning to each non-terminal A £ N a pair (Ha, U) where Ha is a finite 
hypergraph, and Ia : {1,- •■ ,p(A)} Vg A is an injective mapping associating to each position in 
an hyperedge labelled A a vertex in Ha- 

Example 2.2. Figure |2TT] presents an example of a HR-grammar. Formally, it is defined by Sf = ({Z} U 
{A} 2 ,{Vi,V , 2} 1 U{a,d} 2 ,{(Hz,lz),(H A ,lA)} ,Z). Non-terminal Z (resp. A) is the only arity (resp. 2) 
non-terminal symbol; {Vi,^} (resp. {a,d}) are the two colours (resp. arc-labels); hypergraphs Hz, Ha 
and injection Ia are represented in the first part of the figure. For simplicity, V\ denotes the absence of 
colour V\. The injection Ia is used to identify vertices of Ha with vertices of an arc labelled A in the 
rewriting process defined later on. 




Figure 2. 1 : An example of a graph grammar. 



Remark 2.3. Note that Definition ^. 1 I corresponds to the classical definition of deterministic hypergraph 
grammars l2l|5l, since there is exactly one rewriting rule for each non- terminal symbol. Moreover, we 
implicitely assume that terminal symbols have arity one or two (Markov chains are transitions systems, 
thus arities greater than 2 do not make sense in this context). This way, the generated graphs are coloured 
graphs (or transition systems where transitions and states are labelled). 

Let <S = (N, T,R,Z) be a hypergraph grammar. Given A G N a non-terminal, we denote by — > the 

RA 

rewriting relation between hypergraphs with respect to the rule (Ha,Ia) G R- Formally, a hypergraph 

M rewrites into M', written M — ► M', if there exists a hyperarc X = Av\V2 ■ ■ ■ v„ in M such that M' = 

ra ' 

(M — X) UIi(Ha) where h is an injective morphism that maps i(i) to v, and other vertices of Ha to 
vertices outside M. Intuitively, M' is obtained from M by replacing X (of non-terminal label A) with 
Ha. The rewriting relation extends to the complete parallel rewriting relation: the rewriting of each non- 
terminal simultenaously. We write M M' for the complete parallel rewriting of M into M' . In other 

words, all non-terminal hyperedges of M have been replaced in M' using their respective rewriting rules 
in R. The set of all images of a graph M by => is denoted by R[M]. This set contains all isomorphic 

R 

graphs obtained by applying the rules of R to M. For n > 1 , this notation is extended inductively into 
/?' ! [M] = \JMr£Rn-i\M\R[M'], it is the set of all isomorphic graphs obtained after n applications of the 
complete parallel rewriting. 
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Let N and T be sets of non-terminals, respectively terminals. Given h a hypergraph labelled by 
NUT, we denote by [H] the set of terminal arcs and colours in H: [H] = H D (Tz Vh Vh U T\ Vh)- For 
<S = (N, T,R,Z) a HR-grammar, the set of graphs generated by 'S is defined as follows: 



[u n > [H n ] \H = ZA\/n> 0,H n ff„+i} 



?(0 contains graphs which are all isomorphic. 
?m , for each vertex v G Vh, we let Lev(v) 



Note that if H n => H n+U then [H„] C Thus the set 

A graph H is generated by £^ if it belongs to $^' B . Let H G 
be the feve/ at which v is generated. Formally, Lev(v) = min{& | v G [//*]}■ Furthermore, notation Can(v) 

stands for the canonical image of v in the finite set of vertices \J AeN Vh a ■ Assuming /4_ i — ->■ Z2£_ j for 
some AgN and v G Can(v) is the unique vertex in Ha whose image by h is v. When vertex v is 

generated in Ha at the i-th position of an arc labelled by B G N, we write Can(v) = (B, i)a- Observe that, 
since v fljt-i, for each j, v is distinct from IaU)- 

Example 2.4. Figure [2TT1 presents an example of a HR-grammar, Fi gure [231 illustrates . starting from the 
axiom Z, two successive applications of the complete parallel rewriting (which coincides here with the 
rewriting of a single non-terminal) and the iteration of this process. In this example, each application 
of the rewriting rules adds new vertices as well as new arcs to the graph. Observe that the names of the 
vertices (except for vo that is distinguished) are not depicted, since they are not relevant to our purpose. 
Up to renaming of the vertices, there is a unique generated infinite graph. 



MM 




Figure 2.2: Application of successive complete parallel rewritings and the generated graph. 



2.3 Basic Properties and Normal Forms for Regular Graphs 

For any rule (Ha,Ia), we say that the vertices Ia({1, • ■ • ,p(A)}) are the inputs of Ha, and {jy e H A hY{\)eN R ^Y 
are the outputs of Ha- In particular, output vertices belong to non-terminal hyperedges. 

Given a non- terminal A G N, we denote by Succ(A) the set of non-terminals appearing in Ha- 

Given a HR-grammar £f = (N, T,R,Z) and anon-terminal hyperarc X = Av\V2 ■ ■ ■ v p , we introduce no- 
tations R m (resp. R W [X]) to denote a particular graph in & m (resp. in (^[X]) w with &[X] := (N, T,R,X)). 

Let <S = (N,T,R,Z) and = (N' ,T' ,R' ,Z') be two HR-grammars we say that <S' is a colouring of 
£f if, for any graphs H G 5f ra and H' G ^ /<B , there is a graph isomorphism between H and H' which also 
preserves colours of H, and there is a colour in T{ which does not belong to 7\. 

We conclude these preliminaries by giving a normal form for HR-grammars. 

Theorem 2.5. [5 / Any regular hypergraph can be generated in an effective way by a complete outside 
grammar. 
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The complete outside property ensures that the only input vertices that are also outputs are vertices 
of infinite degree. It also implies that each output vertex belongs to a single non-terminal hyperarc. 
This property enables one to identify efficiently grammars having vertices of infinite degree, and it also 
ensures that whenever there is no such vertex, inputs and outputs are distinct. In the sequel we assume 
that all HR-grammars we consider are complete outside. 

2.4 Probabilistic Regular Graphs 

In order to obtain a probabilistic graph from one generated by a HR-grammar, we define, for each HR- 
grammar and each graph H in the counting function # : Vh x T% — ^ N, with #(v, d) = |{v' | v A v'}|, 
that associates with each pair (v, a) the number of a-labelled arcs originating from v. Observe that two 
distinct vertices v and v' in H have identical valuations for # as soon as Can(v) = Can(v'). 

Definition 2.6 (Probabilistic graph grammar). A probabilistic hypergraph grammar (PHR-grammar for 
short) is a pair (5f ,jlt) where 5f = (N,T,R,Z) is a HR-grammar, n : T2 — > [0, 1] is a mapping, and for 
each vertex v G R w the sum of the /i-values of all arcs from v is 1: Y,aeT 2 A 4 (a)#(v,a) = 1. 

Remark 2.7. This definition obviously precludes vertices with infinite out-degree. In fact, it is not 
straightforward to introduce a meaningful definition enabling vertices having infinite out-degree. On the 
contrary, vertices with infinite in-degree are acceptable with this definition. 

Proposition 2.8. Given a HR-grammar 5f and a mapping /i : T<i — > [0, 1], one can decide whether (5f , ju) 
is a PHR-grammar. 

Proof. From Theorem 12.51 we may assume that <S is complete outside. It enables to identify vertices 
of infinite out-degree. Let v be such a vertex, and a a label such that #(v,a) = +00, it forbids (5f ,ju) 
to be a PHR-grammar for any value of jl(a). If there is no such vertex, from Proposition 3.13 (b) of 
l5l . there exists an effective colouring of 'S = (N,T,R,Z) with colours representing the degree of each 
vertex (relative to each label). We produce a colouring representing the exact owf-degree relative to each 
element of T2. There are only finitely many such degrees (from the same proposition, (a)). Now from 
these colours we are able to compute # at each vertex v in the grammar and therefore we may check that 
Lier 2 M(fl)#(v ) fl) = l. □ 

Example 2.9. We consider the graph from Example l2.2l The probabilistic mapping jU, defined by n (a) = 
1 and 11(d) = \, yields a probabilistic regular graph. Clearly the sum of out-going edges is 1 for each 
vertex of the graph. 

2.5 Connection between regular graphs and pushdown automata 

There is a strong connection between regular graphs and configuration graphs of pushdown automata. 
Indeed restricted to finite in- and outdegrees, these graphs coincide: see, e.g., [5, Theorem 5.11]. In 
particular, given a pushdown automaton, the transformation into a graph grammar which generates a 
infinite regular graph isomorphic to the configuration graph of the pushdown system is straightforward 
and may be adapted from the proof of Proposition 5.4 in JH. This proposition states that the suffix graph 
of any rewriting system may be generated by a one rule grammar from the non-terminal. We illustrate 
this construction on the following example. 

Example 2.10. Let us consider the following pushdown system 

a r> 1 1 a a lb, a 

r^fBr r — > Ar r — > Ap BAp^p. 
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To match more closely [5, Proposition 5.4] it is presented as a suffix rewriting system: the state of the 
pushdown automaton is on the top of the stack, and rules are applied to suffixes of the stack. For example, 
when in state r, and whatever the contents of the stack, while reading an a, stack-symbol B is pushed 
and the new state is r 1 . The transformation of this pushdown automaton into a graph grammar goes as 
follows. There is a unique non-terminal X (which, hence, serves as axiom). The vertices of Hx are 
words: each strict suffix (distinct from the empty suffix) of the words appearing in the rewriting rules (in 
the left- and right-hand sides) belongs to the image of ix- Here r, p, r' and Ap are the non-empty strict 
suffixes and they are represented on the top line of the graph Hx- For every stack symbol (here A and B), 
and every non-empty strict suffix, a vertex is formed by the concatenation of the stack symbol and the 
suffix. This yields new vertices, such as Br and all the ones on the bottom line of Hx, but some vertices 
might already be present, as Ap in this example. For each stack symbol, a non-terminal arc, labelled by 
X connects these vertices: Ar,Ap,Ar' ,AAp and Br,Bp,Br* ,BAp, respectively. This construction ensures 
that each left- and right-hand side of the rewriting rules is one vertex. It now suffices to add terminal 
arcs between the vertices according to the rules. For example the a-edge from r to Br 1 encodes the first 
rewriting rule. 



r p r ' JLAp 



H x : l(l) • l(2) . l(3) 



a a 




Br Bp X Br 1 BAp Ar Ar' AAp 



Notice that this construction produces several connected components. Yet, given an initial configuration 
only the connected component (co-)reachable from this configuration will be relevant. 



A similar transformation can be applied to any pushdown automaton in order to obtain a graph gram- 
mar which generates the configuration graph of the pushdown system. This underlines the generality of 
the model of graph grammars. Moreover, we argue the framework of graph grammars is more conve- 
nient than the pushdown automata view. Indeed, transformations presented in Subsection 12.31 on graph 
grammars do not affect the graph they generate, contrary to most transformations on pushdown automata 
that affect the structure of the configuration graph. 

Esparza et al. propose in |H1 a model of probabilistic pushdown automata, derived from pushdown 
automata by assigning weights to rules. The configuration graphs of such systems are infinite state 
Markov chains. Probabilistic pushdown automata and PHR-grammar relate in the same way than push- 
down automata and graph grammars do: the Markov chains defined by both models are the same. More- 
over, any probabilistic pushdown automaton can be turned into a PHR-grammar which generated exactly 
the same infinite state Markov chain. In this sense our model does not generalize the previous model. On 
the other hand, [ 8 ] makes several syntactical assumptions on pushdown automata which do not restrict 
the class of Markov chains, but make it more difficult to manipulate. Transformations of probabilistic 
pushdown automata in order to fit these assumptions may alter the properties of the Markov chain. On 
the contrary, transformations of PHR-grammars do not affect the Markov chain generated. 
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3 Verification of probabilistic regular graphs 

3.1 Markov chains and PCTL 

A (discrete-time) Markov chain is a tuple ^ = (S,so,p) consisting of a (possibly infinite) set S of states, 
an initial state so, and a probabilistic transition function p : S x S — > [0, 1] such that for every state s, 
Y,s'esP( s > s ') = 1- F° r simplicity, we assume the transition system is finitely branching, i.e., in any state 
s there are only finitely many states s' with p(s,s') > 0; the condition Y,s'es p(. s i s ') = 1 is thus well- 
defined. Given a set of atomic propositions AP, a labelled Markov chain ^# = (S,so,p,£) is a Markov 
chain (S,so,p) equipped with a labelling function £ : 5 — > AP. 

Introduced in [9], PCTL is an extention of CTL with probabilities. It can express quantitative proper- 
ties about executions in Markov chains, e.g., with probability 0.9 any sent message will be acknowledged 
in the future. The syntax of PCTL is the following: 

(p ::= tt | a | ->(p\ q> Ay \ X~ p (p \ (pU~ p \j/ 

where a £ AP is an atomic proposition, p £ [0,1] and ~£ {<,<,>,>}■ Operators X~ p and U~ p are 
respectively the probabilistic next-state and until operators and generalise their nonprobabilistic counter- 
parts. Recall the shortcuts in CTL for eventually (F ) and globally (G ): F q> = tt U (p and G (p = ->F -><p. 
Their probabilistic extensions F~ p and G~ p will also be convenient in the sequel. 

Let ^# = (S,so,p,£) be a labelled Markov chain, and s £ S. For a (non-probabilistic) formula of 
CTL, we write F(s |= 0) for the measure of the set of paths in ^# issued from s and which satisfy 0. Note 
that for Vi and V2 sets of states, the set of paths from s satisfying XVi or V\ U V2 is clearly measurable. 
The semantics of a PCTL formula (p over ^# is defined inductively: 

[tt]=s I<pAv] = I<p]nM 

H = G S I a £ ^(s)} (X~P(p]={seS\ F(s |= X [<p]) ~ p} 

h<p] = 5\ W [<pU-P^ = {s€S\F(s |= MUM) ~ p} 

[F-P<p] = {5 £ 5 I P(,s |= F M) ~ p} [G~"<p] = £ S I P(, |= G W) ~ Pi 

and we write s \= (p for s £ [<pj . 

In the following, we will interpret PCTL formulae over labelled Markov chains induced by PHR- 
grammar. Atoms in these formulae will be sets of vertices and will form the set of atomic propositions 
AP. 

Example 3.1. Considering the graph presented in Example l2.2l the probabilistic mapping given in Exam- 
ple 12.91 and predicates V\ and V2 satisfied by vertices labelled by these respective colours, the following 
formulae are of interest: 

• (p\ = V\ AX-2V2: Vertices that satisfy (p\ belong to V\ and with probability greater than |, their 
successors in one step are in V2. In particular, vertices at a fork on the lower line of Figure I2T21 
satisfy (pi. 

2 

• «?>2 = vo A Vi U 3 V 2 : Vertex vo satisfies (pz if the probability of all paths issued from vo that even- 
tually reach V2 passing through vertices of Vi only is greater than |. 

3.2 Qualitative model checking for probabilistic regular graphs 

The qualitative fragment of PCTL only involves the probability thresholds and 1. Let = (N, T,R,Z,ji) 
be a PHR-grammar. Up to isomorphism 8? generates a unique infinite state Markov chain (or ^ 
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when there is no ambiguity on J 2 *). The qualitative model checking problem for probabilistic regular 
graphs is, given a PHR-grammar & with initial vertex vo and a qualitative PCTL formula (p, to answer 
whether in ^tg>, vo |= (p. Mimicking the finite Markov chain approach, the set of vertices satisfying a 
qualitative formula can be effectively computed. 

Theorem 3.2. Let (p be a qualitative PCTL formula, and 2? a PHR-grammar. There is an effective 
colouring in which the set {v G V G \ v |= <p} is identified by a new colour. 

Proof. The proof is by induction on the structure of (p, using the fact that the following sets of vertices 
can be effectively coloured in the graph grammar: {v G V G I P(v,X V) = 1}, {v G V G | P(v,XV) = 0}, 
{v G V G | P(v, Vi UV 2 ) = 1} and {v G V G | P(v, Vi UV 2 ) = 0}. 

Let us start with the two first cases: {v G V G | P(v,X V) = 1} and {v G V G | P(v,XV) = 0}. The 
function Can induces a finite partition on vertices of the infinite Markov chain generated by Two 
vertices with same image by Can have equivalent successors. By hypothesis on the grammar, for every 
vertex generated at level n, all successor vertices are generated between levels n — 1 and n + 1. Hence, if 
v is generated in Ha, it is sufficient to identify in R 2 [A] whether all successors of v belong to V or V. One 
can thus, in the hypergraphs Ha (for each A G N), annotate by colours the vertices which have all their 
successors in V, as well as those which have no successors in V. These colours precisely correpond to the 
sets {v G V G I P(v,X V) = 1} and {v G V G | P(v,X V) = 0}. 

The two other cases {v G V G | P(v,Vi U V 2 ) = 1} and {v G V G | P(v,Vi U V 2 ) = 0} are treated simi- 
larly. We detail here the colouring of {v G V G | P(v, V\ U V 2 ) = 1}. For B G Succ(A) and i < p(B) we let 
R((B,i) A ) = {Aj\B(Bi,Aj) > 0}. We then define inductively the sets: 

• Wo = (// z ny 2 )U{v|Can(v) = (B,i) A and W(5,-)a = 1}, and 

• w„+i =W„U{v|Can(v) = (B,i) A and W(B«)a +LA ; eR((B,o A ) ID) (^> A i) = 1 and R((B, i) A ) C W n }. 
Vertices in Wo are directly winning, either because they already belong to V 2 or because from S ( - in 
context A, the probability to win without decreasing level is 1. Vertices in W n +\ are also almost surely 
winning (i.e. satisfy V\ U V 2 with probability 1) because they are winning without decreasing level (factor 
W(Bi) A ) or firstly decreasing level and then win from Aj with probability 1 (since Ay G W„). 

Clearly, U"=o^« = {v G V G | P(v, Vi U V 2 ) = 1} and the W„'s can be iteratively computed and anno- 
tated in the grammar by colours. □ 

3.3 Probability computation for probabilistic regular graphs 

We now face the problem of computing, given vo an initial vertex in Hz and a CTL formula, the 
probability in of the set of paths starting in vo and satisfying (j>: P^,(vo |= (j>). This can be done 
inductively on the structure of <p, and the difficult part amounts to computing, given V\ and V 2 colours, the 
probability starting in vo to satisfy V1UV2, written P(vo |= V1UV2). This subsection focuses on solving 
this problem. 

3.3.1 Preliminaries and notations 

Without loss of generality we assume that vertices of V\ and V 2 are annotated in the grammar by colours 
(terminals of arity 1) and that vo appears in Hz the hypergraph of the rewriting rule associated to the 
axiom Z of Using the levelwise decomposition of the Markov chain ^M^, we show how to express 
P(vo |= V\ U V 2 ) as a solution of a system of polynomial equations derived from the axiom and the rules. 

The hypotheses we demand on PHR-grammars ensure that the first step of any path issued from a 
vertex of level n either remains at level n or reaches one of the neighbour levels, n — 1 and n + 1 (from 
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Theorem |2.5l it corresponds to restricting to finite degree). This fact will enable levelwise decomposition 
of paths in the Markov chain. 

To compute probabilities in Markov chains generated by PHR-grammars we exploit the regularities 
of the underlying graphs. For v a vertex of ^t@> with Can(v) G Ha, we write ^#[v] for the part of ^#^» 
with underlying graph R W [A] which contains v and no vertices of level Lev(v) — 1. Intuitively, if v has 
been generated by a non-terminal A, we consider the infinite (sub-)Markov chain generated from this 
non-terminal. For two vertices v and V of with Can(v) = Can(v') G Ha, the isomorphism of ^f[v] 
and ./tf[V] ensures that for any CTL formula 0, P ^r v i(v |= (j)) = P^r^i (V |= (j)). In particular, if is the 
formula ViUV 2 , we obtain that: the probability to succeed satisfying ViUV 2 without decreasing level 
is the same from v and from V . The probability to satisfy (V\ \V 2 ) while decreasing level of 1 is also 
independent of the level, provided the initial state corresponds to a fixed canonical representant (S,z)a- 
This motivates the introduction of notations for such probabilities, that are determined by the context and 
are independent of the level. 

Let A,B £ N be. non-terminals such that B G Succ(A). Starting in state v, with Can(v) = (B,i)a, 
each successor state belongs to /? 2 [A], the sub-graph obtained from non-terminal A by two successive 
complete parallel rewritings. Given i < p(B) and j < p(A) we introduce: 

• D(B,-,A 7 ) as the probability from V , with Can(v') = (B, i) A , to reach v such that Lev(v) = Lev(v') — 
1 and v = IaU) satisfying along the path: (Vi \ V 2 ) H (Lev > Lev(v')); 

• W(B,) A as the probability from V, with Can(v') = (B,i) A , to fulfill (Vi n Lev > Lev(v'))U V 2 . 
(Here Lev > k denotes that the current level is greater than a given natural k.) 

As explained before, B(B,-,A ; ) and W(B,)a do not depend on V and v but only on their images by 
Can. Moreover, D(B,,A ; ) expresses the probability to decrease level by one while satisfying a given 
property and W(B,-)a is the probability to win, i.e., to fulfill V1UV2 without decreasing level. This 
justifies the chosen notations. 

The levelwise decomposition of paths is given by vertices belonging (when generated) to non- 
terminal. Thus, given A,B,C,D € such that B,D £ Succ(A) and C G Succ(B), we introduce notations 
for some probabilities that can be computed directly in any portion R 2 [A] of the Markov chain. 

• p{Bj)A is the probability in R 2 [A] from v with Can(v) = {B,i)a to fulfill V1UV2 without visiting 
any V = l A (j) nor V with Can(v') G {(C,k) B , (B,h) A }. 

• p{Bi,Dh)A is the probability in/? 2 [A] from v with Can(v) = (B, i) A to fulfill G (Vi \ V2) and reach V 
with Can(v') = (D,j) A before any v" such that Can(v") G {(B,l) A ,(D,h') A } and Lev(v") = Lev(v). 

• ^(fi,-,A ; ) is the probability in R 2 [A] from vertex v with Can(v) = (B, i) A to reach v' with v' = IaU) 
and Lev(v') = Lev(v) - 1 and satisfy G (Vi \ V 2 ) without seeing any v" G {(C,k) B , (B,l) A , (D,h) A }. 

• ~p*(Bj,Ck)A is the probability in R 2 [A] from v with Can(v) = (B,i) A to reach V with Can(v') = 
(C,k) B satisfying G (Vi \ V 2 ) without visiting any v" = l A (j) nor v" G {(C,k') B , (B,h) A }. 

Intuitively, there are several alternatives for paths starting in v (with Can(v) = (B,i)a) and for which 
V\ U V 2 is not falsified: either they satisfy V\ U V 2 without visiting any vertex at some position on a non- 
terminal hyperarc, or they satisfy G V\ \ V 2 and reach some vertex v' at a given position on a non-terminal 
hyperarc. The above probabilities split these cases according the first V encountered: V can be at the 
level of v (at the h-th position in hyperarc D), or at levels n — 1 (thus of the form IaU)) or n + 1 (at the 
k-th position in hyperarc C). As argued before, p(B{)a, p(Bi,Dj)A, ^{Bi^Aj), and ~~j${Bi,Ck)A can be 
computed directly in /? 2 [A], obtained from Ha, H b , and He for all E G Succ{A) U Succ{B). 
Example 3.3. We compute these probabilities on Example l2.2l /?(A 2 )a = a, p{A\ ,A 2 )a = a, V(^2j M ) = 
d and "^(Ai,A 2 ) A = 0. 
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3.3.2 Computation of P(v |= V x U V 2 ) 

Theorem 3.4. The 0(Bj,Aj) 's and W(B/)a 's satisfy the following equations: 

B(B u Aj) = t(Bi,Aj) +Y d P(B i ,D h ) A -B(D h ,Aj) + J^f(B i7 C k ) A -£B(Q,^) -0(B e ,Aj) (1) 

D h C k B e 

W(Bi) A = p(Bi) A + Y,p(Bi,D h ) A ■ W(D h ) A + Y i ~t(B i ,C k ) A (w(C k ) B + • W(Bj) A ) . (2) 

Moreover, if we add the following constraints: 

• ifBj (V[ \ V 2 ) then B(B { ,Aj) = Ofor every Aj, and 

• ifBj G V 2 then W(S ; )a = 1, and ifB t £ (V l UV 2 ) then W(fi ; -) A = 0; 

the 0(Bi,Aj) 's and W(B,)a 'sform the least solution of this system of polynomial equations. 

Proof. The correctness of Equations Q]and[2]is proved by partitioning the set of paths issued from vertex 
v with Can(v) = (B, im- 
precisely, concerning EquationQ] any path from v with Can(v) = (B,i) A to v' = l A (j) (and Lev(v') = 
Lev(v) — 1) satisfying GVi \ V2 falls in exactly one of the following cases: 

• either it goes directly from v to v' without leaving v's level; 

• or it reaches vertex v" with Can(v") = (D,h) A and Lev(v") = Lev(v), and then goes from v" to V; 

• or it reaches some vertex v" with Can(v") = (C,k)s and Lev(v") = Lev(v) + 1, and then returns to 
v's level at vertex v( 3 ) with Can(v( 3 )) = (B,£) A and from there finally reaches V . 

This case distinction is illustrated on Figure [3j] where plain arrows represent paths in R 2 [A] (as presented 
earlier) and dotted arrows represent recursive probabilities to decrease level. 




Figure 3.1: Illustration of Equation (Q]) for B(B,-,A 7 ). 



For Equation |2l the reasoning is similar. Any path issued from v satisfying Vi U V 2 without visiting 
vertices of level smaller than Lev(v): 
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• either satisfies V\ U V% without visiting any other non-terminals (and hence at v's level) 

• or reaches a vertex V with Can(v') = {D,K)a and Lev(v') = Lev(v) and from then on satisfies 
V\ U V 2 without decreasing level 

• or goes to vertex V with Can(v') = (C,k)s and Lev(v') = Lev(v) + 1, and from there either satisfies 
V\ U V2 without going back to verticesat v's level, or reaches some v" with Can(v") = (B,£)a and 
Lev(v") = Lev(v) and from v" satisfy V\ U V 2 without decreasing level. 

These partitions of the set of paths issued from vertex v with Can(v) = (B,i)a justify Equations Q] and [2] 
The system of equations defines an operator £P : [0, 1]" — > [0, 1]" where n is the number of variables 
appearing in the system. The valuation J^(v) of the variables is obtained by evaluating each equation 
the right-hand side where each variable is substituted with its value in v. This operator is monotonic 
and continuous, and hence admits a unique least fixed-point, which is eventually reached by iterating 
on the null-valuation which assigns to all variables. Note that the convergence towards the least 
fixed-point might require infinitely many iterations. 

To prove that the B(5,-,A ; -)'s and W(B,-)'s form the least solution of the system, we consider the 
probabilities approximated by truncating the paths at length k. Precisely, let B(fi,, Aj) k be the probability- 
mass of 3(Bi,Aj) restricted to paths of length at most k, ; similarly let W{Bi) k A be the probability-mass 
of paths of length at most k in W(5;)a- As k tends to infinity, those probabilities tend to B>(Bi,Aj) and 
W(B ; )a, respectively. It is thus sufficient to prove that, for any k £ N, B(Bi,Aj) k and W(Bj) A are no 
greater than the least solution of the system. This is easily done by induction on k. □ 

Recall that our goal is to compute P(vo |= V1XJV2). This probability can be expressed using the 
B(S ( -,A ; -)'sandW(B ; -) A 's: 

P(v h ^UV 2 ) = p(v )z + £ f(v ,Ai)z W(A/)z, (3) 

A,eSucc(Z) 

where 

• p(vo)z is the probability in Hz from vo to fulfill V\ U V2 without visiting any vertex v' with Can(v') = 
(A,i)z for some A G Succ(Z); 

• ~/^(vo,A;)z is the probability in Hz from vo to V with Can(v') = (A, i)z while satisfying G {V\ \ Vz) 
and without visiting any vertex v" such that Can(v") = {B,j)z (for some B £ Succ(Z)) in between. 

Example 3.5. We illustrate the computation of P(vo |= V\ U V2) on our running example. Since Ca n (vo) = 
(A,l)z and vo ^ V2, p(vq)z = and ~~$(vq,A\) = 1. From Equation |3] we deduce P(vo |= V1UV2) = 
W(Ai)z- Let us detail some steps of the computation. 

W(Ai) z = aW(A 2 )z + a(W(Ai) A + B(Ai,Ai)W(Ai) z + B(Ai,A2)W(A2)z) 
= aW(A 1 ) A + aB(A 1 ,A 1 )W(A 1 ) z , 

since W(A 2 )z = 0. The probability W(A 2 )a is easily computed: W(A 2 ) A = a. Then B(A 1 ,A 1 ) is the 
least solution of a quadratic equation: 

aB(Ai,Ai) 2 -B(Ai,Ai)+acf = 0. 

Letting that a = ~ and d = ~, we get B(Ai,Ai) = 1 - Finally 

W (A0z = aW{M)A - s = t 4? 1 ^W(A0z = -(2V3-3) » 0.31. 

v ' l-aB(A 1 ,A 1 ) (l-a-aB(A 1 ,A 1 ))(l-aB(Ai,Ai)) v ; 3 V ; 
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Note that the exact computation of the solutions of the system may not always be performed. Indeed, 
in general, the equations are polynomials (of arbitrary degree) in the variables. However, similarly as in 
1H, approximate values for the solutions can be computed. 

Theorem 3.6. Let & = (N ,T ,R,Z,ii) be a PHR-grammar, and vq a vertex in Hz- For p e Qn [0, 1] 

and ~G {<,<,>,>}, it is decidable whether P(vo |= V\ U V 2 ) ~ p. Moreover, given < A < 1, one can 
compute Pi , P2 G Q such that p\ < P(vq \= Vi U V2) < P2> and p 2 — P\ < A. 

Proof. Deciding P(v |= V\ U V 2 ) ~ p is equivalent to deciding p(v ) z +XU f eSucc(z) 7^(Vo,Ai)zW(A/) z ~ 
p. Using Equations Q] and [2j the decidability of the first order arithmetics of reals |12] yields the de- 
cidability of our problem. An iterative application of the decision algorithm allows to compute in a 
dichotomic way the desired approximations p\ and p 2 . □ 

3.4 Undecidability of quantitative model checking 

In this subsection, we give a proof of the undecidability of the exact quantitative PCTL model-checking 
problem for PHR-grammars. Since PHR-grammars generalise probabilistic pushdown automata, this re- 
sult is a consequence of the undecidability of quantitative PCTL model-checking for probabilistic push- 
down automata [2]. We however adapt the proof presented in to graph grammars, which, in our 
opinion, enable a simpler exposition. 

The undecidability is proved by a reduction of Post Correspondance Problem (PCP). Recall that an 
instance of the PCP is a sequence of pairs of words ((ui,Vi))i<„ over a fixed alphabet E, and the problem 
is to determine whether there is an integer k, and a sequence (ie)e<k such that «, 2 . . . Uj k = v (1 v ; - 2 . . . vt k . 

The quantitative model-checking problem of PCTL for PHR-grammars is the following: 



Theorem 3.7 ([2]). The quantitative model-checking problem of PCTL for PHR-grammars is undecid- 
able. 

Proof. This result is a consequence of (H but we give here a direct proof. Let ((w/,v;));<„ be a sequence 
of pairs of words on £ = {0, 1}. From this instance of PCP, we define the following PHR-grammar: 
&= (N,T,R,Z,li), where: 

• N = {Z} U {New,- I i < n} 2 ; 

• T = {s, green, red} l U {a,b} 2 ', 

• 11(a) = 0.5, 11(b) = 1; 

and the set R = (Hb, Ib)ben of rewriting rules is depicted below: 



Instance: A PHR-grammar and a PCTL formula (p. 
Question: Is (p valid on ^#^? 



CviOvil) 



C v .(2) C v .(l) O,(0) 



ff NeWj : 





Oii(| «i|) 



C„ ; (2) 0,(1) 0,(0) 
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Colours green, and red label vertices as follows. For each i <n, k < |w,-|, and k' < |v ; -|, 

r (iA-1 Sreen if u t {k) = 1 , f) _ ( green if v,(fc') = 

mA j ~\ red if Bf (Jfe)=0 ' Cv ' l/cj -\ reJ ifv,(£') = l 

Consider the following PCTL formula: 

<p = 5 A (tt U = ? Green) 

where Green and S are atomic propositions corresponding to vertices labelled respectively by green and 
s, terminals of arity 1. We claim that (po is valid on if and only if there is a solution to the Post 
instance ((k;,v;))/<„. 

In the infinite graph generated by SP, each vertex labelled s is connected to the origin (labelled green 
in Hz) via a sequence of w,-'s on the lower branch, and of v,'s on the upper branch (with the same indices). 
Let/=(/ , hi' " -,1m) be a sequence of indices in {1, • • • ,«}, and consider vj the s- vertex corresponding to 
this sequence. The probability to reach red from vj is the following: P(v/ |= ttU Red) = ^(wpath + v path) 
with 

"path =11 , aQ^KD-rt ^W = 0) and v path = £ £ 2 ( £ Jv ;/ | )+ . ^( fc/ ) = 0- 

The only situation where P(v/ |= ttU Red) = \ (and hence P(v/ |= ttU Green) = I) occurs when the 
same sequence of letters appear in w pa th and v pa th (from the unicity of the binary expansion). □ 

4 Conclusion 

In this paper we introduced probabilistic regular graphs, as graphs generated by graph grammars where 
terminal arcs are labelled with probabilities. Results concerning the model-checking of probabilistic 
pushdown automata extend to this context. Precisely, both the approximate PCTL and qualitative PCTL 
model checking problems are decidable, whereas the exact quantitative model-checking problem is un- 
decidable. 

We believe that our model of PHR-grammars offers a major benefit compared to pushdown systems: 
it focuses on structural aspects whereas configurations graphs of pushdown automata emphasise com- 
binatorial aspects. Furthermore in order to identify classes of infinite state systems with a decidable 
quantitative PCTL model checking we believe that structural restrictions on the grammar might prove 
worth studying. A natural extension of our work is to extend the positive results to graphs where infinite 
in-degree in allowed. Another research direction is to try to climb up the Caucal hierarchy, like [4], and 
pursue our work on higher-order pushdown systems. 

References 

[1] T. Brazdil, V. Brozek, J. Holecek & A. Kucera (2008): Discounted Properties of Probabilistic Pushdown 
Automata. In: Proceedings of the 15th International Conference on Logic for Programming, Artificial Intel- 
ligence, and Reasoning (LPAR'08), Lecture Notes in Computer Science 5330, Springer, pp. 230-242. 

[2] T. Brazdil, A. Kucera & O. Strazovsky (2005): On the Decidability of Temporal Properties of Probabilistic 
Pushdown Automata. In: Proceedings of the 22nd Annual Symposium on Theoretical Aspects of Computer 
Science (STACS'05), Lecture Notes in Computer Science 3404, Springer, pp. 145-157. 



90 



Probabilistic regular graphs 



[3] J. R. Biichi (1964): Regular Canonical Systems. Archiv fur Mathematische Logik und Grundlagenforshung 
6, pp. 91-111. 

[4] A. Carayol & S. Woerhle (2003): The Caucal Hierarchy of Infinite Graphs in Terms of Logic and Higher- 
Order Pushdown Automata. In: Proceedings of the 23rd Conference on Foundations of Software Technology 
and Theoretical Computer Science (FSTTCS'03), Lecture Notes in Computer Science 2914, Springer, pp. 
112-123. 

[5] D. Caucal (2007): Deterministic graph grammars, Texts in logics and games 2, pp. 169-250. Amsterdam 
University Press. 

[6] D. Caucal & T. Knapik (2001): An internal presentation of regular graphs by prefix-recognizable ones. 
Theory of Computing Systems 34(4). 

[7] B. Courcelle (1990): Graph rewriting: an algebraic and logic approach, Handbook of Theoretical Computer 
Science B: Formal Models and Semantics, pp. 193-242. Elsevier. 

[8] J. Esparza, A. Kucera & R. Mayr (2006): Model Checking Probabilistic Pushdown Automata. Logical 
Methods in Computer Science 2( 1). 

[9] H. Hansson & B. Jonsson (1994): A logic for reasoning about time and reliability. Formal Aspects of 
Computing 6(5), pp. 512-535. 

[10] A. Kucera (2006): Methods for Quantitative Analysis of Probabilistic Pushdown Automata. Electronic Notes 
in Theoretical Computer Science 149(1), pp. 3-15. 

[11] D. Muller & R Schupp (1985): The theory of ends, pushdown automata, and second-order logic. Theoretical 
Computer Science 37, pp. 51-75. 

[12] A. Tarski (1951): A Decision Method for Elementary Algebra and Geometry. University of California Press, 
Berkeley. 



